United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
I nilid Stall-, Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 



APPLICATION NO. 



10/734.793 



FILING DATE 



7590 

Frederick W. Gibb, III 
McGinn & Gibb, PLLC 
Suite 304 

2568-A RivaRoad 
Annapolis, MD 21401 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



JP920030161US1 



HOAR, COLLEEN A 



PAPER NUMBER 



DELIVERY MODE 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



l/ffflrC? nVrliUli Otfff Iff ids y 


Application No. 

10/734,793 


Applicant(s) 

AGRAWAL ET AL. 


Examiner 

Colleen Hoar 


Art Unit 

3622 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
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DETAILED ACTION 

Status of Claims 

Claims 1 , 3-9, 11-18 are pending in the application. Claims 1,9,11-15,17-18 are 

amended. Claims 2, 10, are cancelled. Claims 3-8,16 as previously presented. 

Examiner's note: Examiner has pointed out particular references contained in the prior art of record in the 
body of this action for the convenience of the Applicant. Although the specified citations are 
representative of the teachings in the art and are applied to the specific limitations within the individual 
claim, other passages and figures may apply. Applicant, in preparing the response, should consider fully 
the entire reference as potentially teaching all or part of the claimed invention, as well as the content of 
the passage as taught by the prior art or disclosed by the Examiner. 



Claim Rejections - 35 USC §112 

Claims 1 , 3-9, 11-18 are rejected under 35 U.S.C. 1 1 2, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

The independent claims mention customer activities across multiple channels upon 
which promotions are targeted and delivered, however it is unclear how a one-way 
channel such as a catalog or direct marketing can be used in this way. 

A response to a direct marketing piece has not been addressed. In addition, 
when a customer responds to a catalog, he/she does so by switching from the one 
way catalog channel to a telephonic or web channel upon which an offer is made. 
This subject matter continues to be indefinite. 



Claim Rejections - 35 USC § 102 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 
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Claims 1 , 4-7, 9, 1 2-1 7, rejected under 35 U.S.C. 1 02(b) as being anticipated by Abe 
et al., (6055513) hereinafter referred to as Abe . 

Claim 1, 4-7, 9, 12-15,17, Abe discloses storing customer transaction data including 
stimulus-response history data, a customer profile cache for receiving an output of the 
customer transaction cache and storing current customer profile data, and a customer 
relationship management system, for receiving an output of the customer profile cache 
and customer relationship management rules, and execute the action specified by a 
rule that is identified as applicable. In this system, for example, the customer 
relationship management rules may be automatically generated based on the customer 
transaction data. (Page 3, Para 29); providing customer data may include selecting 
customer data by accessing a number of channel specific databases, and forming an 
effective joining of data using a form of customer identification as a key. (Page 2-3, 
Para 25); sequential decision making (e.g., sequential cost-sensitive decision making) 
for customer relationship management. The inventive method includes providing 
customer data (e.g., consumer data, client data, donor data, etc.) comprising stimulus- 
response history data, and automatically generating actionable rules based on the 
customer data. Further, automatically generating actionable rules may include 
estimating a value function using reinforcement learning (e.g., reinforcement 
learning and dynamic programming). For example, estimating a value function may 
include value iteration. (Page 2, Para 14); Further, the actionable rules may be 
generated using reinforcement learning based on a Markov Decision Process model 
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with function approximation. For example, the batch reinforcement learning with function 
approximation may be based on Q-learning and/or sarsa-learning. For example, the 
batch reinforcement learning may be based on training data including sequences of 
states, actions, and resulting rewards. (Page 2, Para 17) using the customer data to 
output instance-in-time targeting rules [before executing said request](Page 2, 
Para 18) Further, the inventive method may be applied to cross-channel optimized 
marketing (CCOM). For example, the inventive method may include providing customer 
data including stimulus-response history data from a plurality of channels (e.g., 
"multiple" channels), integrating the customer data, and automatically generating 
channel specific actionable rules based on the customer data. For instance, in this 
case the method may optimize (e.g., nearly optimize) cross-channel cumulative 
profits. (Page 2, Para 19); providing customer data may include generating a sequence 
of event data which may include a customer's demographic features, and features, 
derived from the customer's stimulus-response history data, which collectively 
reflect the state of said customer at a certain point in time, an action (e.g., a 
marketing action) taken at or around that time by an entity (e.g., a seller such as a 
retailer), a response taken by the customer at or around a time of an event, (Page 
2, Para 20); optimization of sequences of marketing actions, as now the actions are 
not only taken at different points in time, but also across different channels over 
time. (Page 12, Para 179); FIG. 24 illustrates one possible infrastructure for the present 
invention including Cross Channel Optimized Marketing. For example, in this example, 
the channels include the web, kiosk, direct mail, a call center, and a store. The 
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rules from the customer relationship management (CRM) rule base are applied to 
operational transactions [ requests and replies] to transform into customer 
profiles (e.g., operational and historical) at a point in time. The customer lifetime 
value is modeled to predict customer behavior. These models are transformed 
into situation-action rules which may be used to update the CRM rule base. (Page 
14, Para 197); Further, blocks 1560,1565,1575, and 1580 represent an exemplary 
embodiment of an operational CRM system for utilizing the targeting rules that are 
produced by 

the invention. This operational CRM system is depicted as having its own 
transaction (1580) and profile (1565) data caches in order to enhance its run-time 
execution speed by reducing the number of queries that would otherwise have to be 
issued against the main database (1505) if the data caches were not present. (Page 12 
,Para 177); The system 1400 also includes a processor 1420 for automatically 
generating actionable (e.g., targeting) rules based on the customer data. (Page 12, 
Para 167); Further, providing customer data may include selecting customer data. For 
example, a value estimation may repeatedly call a data selection module one or more 
times per each iteration of a value iteration. In addition, the data selection module may 
access a customer and transaction history data set stored in a data storage device, and 
use a selection criteria to select a restricted subset for use in a value function 
estimation (Page 2, Para 23). [Examiner equates a value estimation to a belief or 
probability assignment of a customer action as applied in calculations and algorithms of 
the reference.] 
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Claim 8, 16, Abe discloses As each action is performed, Equation 5 is first 
used to update the Q-value for the state just visited, and Equation 6 is then used to 
update the action that is to be taken (with probability (1-E)) the next time that state is 
visited. (Page , Para 91). 
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Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 3, 11, 18 rejected under 35 U.S.C. 103(a) as being unpatentable over Abe in 
view of Reiser et al. (6125339). 

Claims 3, 1 1 , 1 8 : Abe discloses a customer relationship system that integrates 
customer profiles across multiple channels and generates operational rules for 
upselling, promotions, offers, etc. using statistical processes such as Markov Decision 
Process, regression analysis, Sarsa-learning, Q-learning. Abe does not explicitly 
mention Dempster Orthogonal Sum belief processes, however, this approach is a basic 
probability assignment process which is an approach taken by the Abe invention. 
Reiser teaches the Dempster-Schafer belief process,- a method of fuzzy logic for 
automatically learning belief functions "including the steps of gathering information 
representative of an object or event; creating a set of basic probability assignments 
based on said set of information; creating combinations of said basic probability 
assignments; measuring an error present in said basic probability assignments and said 
combinations of basic probability assignments; calculating updates of said basic 
probability assignments and said combinations of basic probability assignments based 
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on said error; and modifying said basic probability assignments and said 
combinations of basic probability assignments with said updates. (Col 2, lines 6- 
23). It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify the invention of Abe to use the Dempster-Shafer basic 
probability assignment method in order to predict which promotion or upsell is most 
likely to be accepted by a customer. 



Response to Arguments 

Applicant's arguments filed January 29, 2009 have been fully considered but they 
are not persuasive. 

Applicant argues that (re: the 112 rejection) examiner is reading limitations into 
the claim. Examiner disagrees. Examiner is confused how the (claimed) catalog or the 
(claimed) direct marketing can be used as a two-way channel as applicant states. 
Examiner is not however confused how an on-line computer can be used to accomplish 
2-way communications. 

Abe references multiple behavior profiles or multiple channels to generate 
channel specific offers using various statistical and probability processes to make a 
prediction. In addition, Abe discloses simulating online reinforcement learning with a 
particular policy by electing to use just those data that conform to the policy. (Page 8, 
Para 110) and therefore anticipates applicant. Therefore, the inventors have developed 
cross-channel optimized marketing (CCOM) technology. Specifically, as shown in FIG. 
19, the inventors are developing CCOM for customer lifetime value maximizing by 
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addressing at least three core technical issues, (1) scalability for integrating large multi- 
channel data, (2) optimizing rules across multiple channels and over customer's life 
time, and (3) mapping rules into channel-specific actions. 

A key feature of the present invention (e.g., including CCOM) is that it achieves 
desired cross-channel customer behavior via optimized channel-specific targeted 
marketing rules. For instance, the present invention may leverage cross-channel 
historical data on customer buying behavior, channel behavior, and promotion-response 
behavior. Further, it may automatically generate channel specific targeting rules, 
optimized across all channels. In addition, it may automatically generate instance-in- 
time targeting rules [before executing said request], optimized for sequence of targeting 
actions. (Page 13, Para 192-193). The invention of Abe overcomes dealing with large 
data histories for each transaction and therefore anticipates reinforcement learning 
online. 

Reiser uses the Dempster Orthogonal Sum approach for combining data from 
different sources to make a prediction which includes updating said probability 
assignments from those stored in history among multiple combinations [ or channels]. 
This is an obvious substitution algorithms included in Abe . 
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Conclusion 

THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Colleen Hoar whose telephone number is (571)270- 
3447. The examiner can normally be reached on Monday- Thursday 7:30-5:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Eric Stamber can be reached on 571-272-6724. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Jeffrey D. Carlson/ 

Primary Examiner, Art Unit 3622 



/C. H./ 6/4/2009 
Examiner, Art Unit 3622 



Colleen Hoar 
Examiner 
Art Unit 3622 



